… is set

…ead-only DB

Add Replica1 to ecto_repos when DATABASE_READ_ONLY_API_URL is configured,
but skip migrations for read-only repos. This allows:

- Replica1 connection pool to be started by Ecto supervision
- Read queries to route through Replica1 via Repo.replica()
- Migration tasks to skip read-only replicas automatically

Changes:
- Modified release_tasks.ex to filter read-only repos during migrations
- Modified config_helper.exs to include Replica1 in ecto_repos conditionally
- Added is_read_only_repo?/1 helper to check repo.config[:read_only]

Fix syntax error in is_read_only_repo?/1 helper.
repo.config is a function and must be called with parentheses.

Replica1 is started by Explorer.Application supervision tree directly.
This prevents migration attempts on read-only database while still
allowing read query routing via Repo.replica().

Add Replica1 to ecto_repos only during runtime (not during migrations).
Uses RELEASE_COMMAND env var to detect migration tasks and exclude Replica1.

Move Replica1 addition from config_helper.exs to runtime.exs.
This allows runtime environment variable evaluation for conditional
Replica1 inclusion based on DATABASE_READ_ONLY_API_URL and RELEASE_COMMAND.

…eplica1

… exists

RELEASE_COMMAND is set to 'eval' when running bin/blockscout eval commands,
so we need to explicitly check if it equals 'migrate' to exclude Replica1
only during migrations, not during normal runtime eval commands.

Instead of runtime conditional logic that doesn't work in compiled releases,
use a simpler approach:
- config_helper.exs: Always include Replica1 in ecto_repos (static)
- release_tasks.ex: Filter replicas at runtime by checking module name
- runtime.exs: Remove non-working conditional ecto_repos logic

This follows Ecto best practices and ensures:
- Replica1 is always in supervision tree and ecto_repos
- Migrations skip Replica1 by checking module name for 'Replica'
- No complex runtime config evaluation needed

  - Add distribution for transaction_forks, transaction_actions, signed_authorizations, pending_transaction_operations, block_rewards
  - Restore foreign keys for all distributed tables
  - Fix conflict_target in logs.ex and token_transfers.ex to match PRIMARY KEYS
  - Add missing PRIMARY KEY to transaction_forks table
  - Update MONAD_FORK_CHANGES.md with correct Citus-only architecture

…n with Citus distribution

- Changed conflict_target from [:uncle_hash, :index] to [:hash, :index] to match the new PRIMARY KEY.
- Updated sorting order in ordered_changes_list to prioritize the distribution column (hash) for optimal Citus performance.
- Addressed production error related to distributed queries with FOR UPDATE/SHARE commands.

- Changed the default_on_conflict function to use :replace_all instead of a query-based approach.
- This change resolves issues with SELECT FOR UPDATE commands that are incompatible with Citus distributed tables.
- Addresses production errors related to distributed queries, ensuring smoother operation with Citus architecture.

- Updated the default_on_conflict function to use {:replace_all_except, [:hash, :index]} to avoid SELECT FOR UPDATE issues with Citus distributed tables.
- This change explicitly excludes PRIMARY KEY columns from updates, aligning with Ecto best practices and ensuring efficient handling of conflicts.
- Enhances compatibility and resolves production errors related to distributed queries.

…ibility

- Changed the default_on_conflict function to use :nothing, avoiding row locking issues with PostgreSQL's ON CONFLICT DO UPDATE, which is incompatible with Citus distributed tables.
- Updated conflict_target from [:uncle_hash, :index] to [:hash, :index] to align with the PRIMARY KEY and ensure proper handling of distributed queries.
- Adjusted sorting order in transaction_forks to prioritize the distribution column (hash) for optimal performance.
- This addresses critical production errors related to FOR UPDATE/SHARE commands in distributed environments.

…lity

- Eliminated the "FOR NO KEY UPDATE" lock in the fork_transactions function to resolve production errors related to FOR UPDATE/SHARE commands in distributed environments.
- This change allows the subsequent update_all to handle updates without explicit locking, enhancing compatibility with Citus distributed tables.
- Addresses critical issues arising from transaction updates during block reorganizations.

…mpatibility

- Added a query to set Citus multi-shard modify mode to 'sequential' to prevent parallel modification errors on the replicated 'tokens' table.
- This change enhances compatibility with Citus distributed tables during updates.

…compatibility

- Updated the method to use Ecto.Adapters.SQL.query for setting Citus multi-shard modify mode to 'sequential', ensuring it executes within the same transaction as update_all.
- This change enhances the reliability of updates on the replicated 'tokens' table by preventing parallel modification errors.

…ssues

Removed FOR UPDATE/FOR NO KEY UPDATE locks, ctid-based JOINs, and subquery JOIN patterns that are incompatible with Citus distributed tables.

**Critical Fixes (Phase 1 - ctid patterns):**
- blocks.ex: delete_address_coin_balances() - Removed FOR UPDATE lock and subquery JOIN
- blocks.ex: delete_address_token_balances() - Removed ctid-based JOIN (SHARD-LOCAL, incompatible)
- blocks.ex: delete_address_current_token_balances() - Removed ctid-based JOIN
- Migrator: delete_zero_value_internal_transactions.ex - Removed ctid pattern
- Migrator: reindex_duplicated_internal_transactions.ex - Removed ctid pattern

**High Priority Fixes (Phase 2 - distributed table locks):**
- internal_transactions.ex: acquire_blocks() - Removed FOR NO KEY UPDATE on reference table
- internal_transactions.ex: acquire_pending_internal_transactions() - Removed FOR UPDATE on distributed table
- internal_transactions.ex: acquire_transactions() - Removed FOR NO KEY UPDATE on distributed table
- transactions.ex: discard_blocks_for_recollated_transactions() - Removed subquery JOINs, use direct WHERE IN
- blocks.ex: lose_consensus() - Refactored to multi-step approach without subquery JOINs (affects 3 tables: blocks, transactions, token_transfers)

**Optimization (Phase 3):**
- blocks.ex: update_block_second_degree_relations() - Direct UPDATE without subquery
- tokens.ex: update_holder_counts_with_deltas() - Removed subquery IN, use direct WHERE IN

**Root Cause:**
PostgreSQL crashes were caused by Citus rejecting FOR UPDATE/FOR NO KEY UPDATE locks and ctid operations on distributed tables. These patterns worked with vanilla PostgreSQL but fail with Citus distributed architecture where:
- ctid is SHARD-LOCAL (not global across distributed shards)
- FOR UPDATE locks require special handling in multi-shard contexts
- Subquery JOINs with locks trigger Citus incompatibility errors

**Impact:**
- Eliminates XX000, 0A000, 40P01 Citus errors
- Stops PostgreSQL restart loop caused by query failures
- Improves query performance by removing unnecessary locking
- Maintains data consistency with sequential mode where needed (tokens reference table)

**Note:** INDEXER_DISABLE_REPLACED_TRANSACTION_FETCHER already enabled in production config

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

Remove FOR NO KEY UPDATE lock and subquery JOIN pattern from update_transactions() function that was causing continuous 0A000 errors in production logs.

**Error Pattern (56 occurrences in 30 minutes):**
```
ERROR 0A000 (feature_not_supported)
could not run distributed query with FOR UPDATE/SHARE commands
HINT: Consider using an equality filter on the distributed table's partition column.
```

**Root Cause:**
- addresses.ex:update_transactions() was missed in previous Citus fix commit (38c7453)
- Function uses FOR NO KEY UPDATE lock on distributed 'transactions' table
- Citus rejects FOR UPDATE/SHARE locks on distributed queries without partition column equality filter

**Affected Operations:**
- Contract code indexing (created_contract_code_indexed_at updates) - ~40 errors/30min
- Signed authorization status updates - ~16 errors/30min

**Fix:**
- Removed FOR NO KEY UPDATE lock from transaction query
- Removed subquery JOIN pattern, use direct WHERE IN
- transactions is distributed by 'hash' - direct filter is Citus-compatible

**Impact:**
- Eliminates all remaining 0A000 errors in production
- Contract code indexing will now succeed
- Signed authorization status updates will work correctly

**Testing:**
Production logs showed this exact query failing:
```sql
UPDATE "transactions" AS t0 SET "created_contract_code_indexed_at" = $1
FROM (SELECT ... FROM "transactions" AS st0
      WHERE (st0."created_contract_address_hash" = ANY($2))
      ORDER BY st0."hash" FOR NO KEY UPDATE) AS s1
WHERE (t0."hash" = s1."hash")
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

Remove order_by clause from update_transactions() as Ecto.update_all does not support order_by expressions.

**Error Pattern (137 errors/hour in production):**
```
Ecto.QueryError: `update_all` allows only `with_cte`, `where` and `join` expressions.
Error found in query: order_by: [asc: t0.hash]
```

**Root Cause:**
- Previous fix (5d13a47) removed FOR NO KEY UPDATE lock
- But kept order_by: t.hash in the update_all query
- Ecto.update_all explicitly does NOT support order_by clause
- Only allows: with_cte, where, join expressions

**Impact:**
- Contract code indexing was completely broken (137 errors/hour)
- Signed authorization status updates were failing
- All created_contract_code_indexed_at updates failed

**Fix:**
- Removed order_by: t.hash from update_all query
- Ordering is already handled by ordered_created_contract_hashes being pre-sorted (line 262)
- UPDATE order doesn't affect correctness, only determinism of results

**Testing:**
Production logs showed exact error:
```
Task terminating
** (Ecto.QueryError) `update_all` allows only `with_cte`, `where` and `join` expressions
```

**Status After This Fix:**
- ERROR 0A000 (FOR NO KEY UPDATE): ELIMINATED
- Ecto.QueryError (order_by): WILL BE ELIMINATED
- Contract code indexing: WILL WORK

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

…ashing

…arallel modification errors on the replicated 'tokens' table.

…arallel modification errors on the replicated 'token_instances' table.

…igin handling

- Introduced BlockOnDemand and TransactionOnDemand fetchers to the Indexer application.
- Updated runtime configuration to include settings for on-demand block and transaction fetchers, including timeouts and environment variable parsing

…support

- Updated the block fetcher to support multiple archive RPC URLs for improved load balancing.
- Refactored the configuration to accept a comma-separated list of archive RPC URLs.
- Implemented round-robin selection of archive URLs to optimize fetching performance.
- Adjusted internal methods to handle state management for the new URL selection logic.

- Changed the configuration to use a list for archive RPC URLs instead of a single URL.
- This adjustment supports the round-robin fetching mechanism introduced in the previous commit.

…logic

- Updated the transaction fetcher to utilize a list of archive RPC URLs for improved load balancing.
- Implemented round-robin selection of archive URLs to optimize transaction fetching.
- Adjusted internal state management to accommodate the new URL selection mechanism.

- Updated the block transformation logic to include a consensus flag for each block.
- This change ensures that all transformed blocks are marked with a consensus status, improving data consistency.

…arameters

- Modified the fetch_receipts function to accept full transaction parameters, including :gas and :hash, instead of just transaction hashes.
- This change aligns with the requirements of the EthereumJSONRPC.fetch_transaction_receipts function, improving data handling.

- Enhanced the on-demand block and transaction fetchers to implement a retry mechanism for fetching data from multiple RPC URLs.
- Updated error handling to differentiate between retryable and non-retryable errors, improving robustness in data fetching.
- Introduced a method to calculate the maximum number of RPC attempts based on the available archive URLs.

…onfigurable

- Updated the fetch_receipts function to use configurable batch size and concurrency values from the application environment.
- Default values are set to 5 for batch size and 3 for concurrency, which can be overridden via environment variables.